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Abstract 

When instrument meteorological conditions 
(IMC) restrict operations to Instrument Flight Rules 
(IFR) at non-towered, non- radar airports, air traffic 
control (ATC) uses procedural separation that constrains 
operations to only one approaching or departing aircraft 
at a time - the “one -in/one -out” paradigm. The Small 
Aircraft Transportation System (SATS) breaks the one- 
in/one-out paradigm and expands capacity by allowing 
multiple, simultaneous operations while achieving a level 
of safety equal to today’s system. The concept that 
achieves this goal is SATS "Higher Volume Operations” 
(HVO). Characteristic to SATS HVO is the 
establishment of a Self- Controlled Area (SCA), which 
would be activated by ATC around designated non- 
towered, non- radar airports. During periods of poor 
visibility, SATS pilots would take responsibility for 
separation assurance between their aircraft and other 
similarly equipped aircraft in the SCA. Using onboard 
equipment and simple instrument flight procedures, they 
would then be better able to approach and land at the 
airport or depart from it. This concept would also require 
a new, ground-based automation system, the Airport 
Management Module (AMM) typically located at the 
airport that would provide appropriate sequencing 
information to the arriving aircraft. 

This paper provides an analysis of Flight 
Technical Error (FTE) from recent SATS experiments, 
called the Higher Volume Operations (HVO) Simulation 
and Flight experiments, which NASA conducted to 
determine pilot acceptability of the HVO concept for 
normal operating conditions. Reported are FTE results 
from simulation and flight experiment data indicating the 
SATS HVO concept is viable and acceptable to low- time 
instrument rated pilots when compared with today’s 
system (baseline). Described is the comparative FTE 
analysis of lateral, vertical, and airspeed deviations from 
the baseline and SATS HVO experimental flight 
procedures. Based on FTE analysis, all evaluation 
subjects, low-time instrument-rated pilots, flew the HVO 
procedures safely and proficiently in comparison to 
today’s system. In all cases, the results of the flight 
experiment validated the results of the simulation 
experiment and confirm the utility of the simulation 
platform for comparative Human in the Loop (HITL) 
studies of SATS HVO and Baseline operations. 


Introduction 

Americans have come to depend on the United 
States’ National Airspace System (NAS) for the 
efficient and rapid movement of people, goods, and 
services. In 2000, more than 670 billion revenue 
passenger miles were flown [1]. Commercial air 
transport service has become so important that any 
major disturbance in its service is met by public 
outcry. 

While the current system of hub and spoke 
operations has served its purpose well, it is 
beginning to reach a capacity plateau. Due to the 
increasing demand on the system and with only 
modest potential gains in the number of flights, the 
system will reach gridlock within the next 10-15 
years [1,2]. Additionally, most airlines use the more 
economical hub and spoke system which causes 
people to travel significantly farther or longer to get 
to their destination. Nearly 70% of domestic air 
travelers are forced to fly through fewer than 35 of 
the United States' more than 18,000 landing 
facilities. These intermediate stops and layovers 
dramatically increase a traveler’s overall door-to- 
door trip time. The rising success of air carriers and 
air charter services that specifically target more 
point-to-point travel provides evidence that people 
and businesses are seeking greater mobility through 
more convenient alternatives for air service [3]. 

Through the Small Aircraft Transportation 
System (SATS) Project, NASA, the FAA, and the 
National Consortium for Aviation Mobility have 
been exploring the feasibility of increasing personal 
mobility and system capacity by expanding access 
to thousands of underutilized smaller airports across 
the United States. Many of these airports lack 
control towers and lie outside air traffic control 
(ATC) radar coverage, but do provide a unique 
potential for convenient access to small cities and 
business communities. New, small, efficient aircraft 
being developed by companies such as Honda, 
Avocet, Cessna, Diamond, Eclipse, Safire, Cirrus, 



Lancair, Adam Aircraft, and others are touted to 
provide point-to-point air-charter service and make 
use of these small airports. Several air charter 
businesses are planning to use these new aircraft to 
provide their customers with point-to-point service. 

When instrument meteorological conditions 
(IMC) restricts operations to Instrument Flight 
Rules (IFR) at non-towered, non-radar airports, 
ATC uses procedural separation that restricts 
operations to only one approaching or departing 
aircraft at a time - the “one-in/one-out” paradigm. 
While procedural separation is safe, it severely 
limits the operational throughput at these airports. 
Air charter operators might be compelled to use 
these airfields if the IMC operational efficiency can 
be improved. SATS breaks the one- in/one -out 
paradigm and expands capacity by allowing 
multiple, simultaneous operations while achieving a 
level of safety equal to today’s system. The 
concept of operations (CONOPS) that achieves this 
goal is termed SATS “Higher Volume Operations” 
(HVO). 

SATS HVO CONOPS Overview 

Key to this concept is the use of a newly 
defined area of flight operations called a Self- 
Controlled Area (SCA), established during periods 
of IMC around “SATS designated airports” (i.e., 
non-towered, non-radar airports). This concept is 
based on a distributed decision-making 
environment that assumes the majority of the 
decision-making responsibility would remain with 
the pilot because it would provide pilots with the 
necessary procedures, tools, and information to 
enable safe operations within the SCA. 

Within the SCA, pilots, using advanced 
airborne systems, would have the ability and 
responsibility to maintain separation between 
themselves and other similarly equipped airplanes. 
Aircraft operating in this airspace would need 
special avionics, e.g., automated dependent 
surveillance-broadcast (ADS-B), a two-way data 
link, and appropriate self-separation tools in order 
to participate. This concept would also require a 
new, ground-based automation system, the airport 
management module (AMM), typically located at 
the airport that would provide appropriate 
sequencing information to the arriving aircraft. The 
AMM provides an arrival sequence and broadcasts 


the total number of arriving aircraft in the SCA. It 
does not, however, provide separation, altitude 
assignments, or sequence departures. 

This proposed operational concept emphasizes 
the integration with the current and planned near- 
term NAS. Additionally, the focus of the underlying 
design approach was on simplicity from both a 
procedural and a systems requirements standpoint. 

It was also assumed that any additional ATC 
workload must be minimized, and enroute 
procedures must be compatible with today’s ATC 
system. 

A joint NASA Langley Research Center and 
FAA Technical Center simulation study focused on 
the SATS HVO and ATC transitions (i.e., SCA 
airspace design, and controller-pilot SCA transition 
procedures) to ensure that additional ATC workload 
is minimized and that SATS HVO integrates with 
today’s ATC system. Controller acceptance of 
HVO has been positive [9]. 

The SATS HVO concept is a starting point or 
“template” for additional designs and analyses. To 
date, the development focus has been on providing 
an operational concept that was safe, would enable 
more than one operation at a time, and would not 
require significant ground infrastructure costs or 
improvements. 


GPS-T instrument approach procedures were 
chosen as a basis for this concept, although other 
instrument approach procedures could be used. 



Figure 1. SATS HVO Example 

Many of the features of the GPS-T based 
SATS HVO concept are depicted in Figure 1. 
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SATS arrivals (Red and Blue aircraft) to the Initial 
Approach Fixes (IAFs) with alternating missed 
approaches, and departures (Green and Purple 
aircraft) to the Departure Fixes (DFs) are depicted 
in a “snapshot” in time: 

• Blue - entering the SCA having coordinated 
descent with ATC because the AMM provided: 
“lateral entry (no other aircraft assigned to 
Cathy), follow none, missed approach Cathy” 
(missed approach depicted as blue dashed path), 

• Red - having arrived by IFR clearance to the 
transition fix at 4000ft, the AMM provided: 
“vertical entry (3000ft at Cathy is open), follow 
blue aircraft, missed approach to Annie,” 
(missed approach depicted as red dashed path) 

• Purple - departing SCA via departure 
procedure and contacting ATC, 

• Green - released by ATC to depart; holding 
short and using on-board tools to find open slot 
in arrival stream to take the active runway and 
depart. 

Aircraft arriving into the SATS airport will be 
under ATC clearance according to an IFR flight 
plan to a transition fix above the SCA. The 
transition fix is also the initial approach fix on a 
GPS-T instrument approach procedure. Prior to 
reaching the transition fix, the pilot would request a 
landing assignment from the AMM. The AMM 
responds with the SCA entry procedure (standby, 
vertical or lateral), relative sequence information 
(follow <Callsign>), and missed approach hold fix 
assignment (Annie or Cathy). The AMM only 
tracks arrivals and missed approach aircraft, not 
departures, and thereby allows up to four arriving 
aircraft in the SCA before denying entry (issuing a 
“standby”). Based on sequence info, and following 


the HVO procedure to “descend to lowest available 
altitude,” pilots are deconflicted from up to three 
other arriving aircraft (i.e., the AMM reserves space 
for up to four aircraft at the IAFs). 

Pilots given a “standby” sequence can track the 
number of aircraft in the SCA to estimate their 
delay as they continue to their clearance limit and 
establish a standard hold above the SCA at the 
transition fix. When the pilot gets an AMM entry 
message with sequence and missed approach 
information, the pilot is assured an opening at 
3000ft and will request descent from ATC. The 
pilot can then determine if further descent to the 
2000ft hold is prudent by following the “lowest 
available altitude” procedure at the IAF, (clearing 
for traffic below is the pilot’s self- separation 
responsibility in the SCA). A missed approach hold 
slot is also guaranteed by the AMM, so a pilot 
going missed would then climb to the “lowest 
available altitude” back at the IAF and would then 
be sent a new arrival sequence. 

Pilots will initiate their approach once 
adequate spacing behind the lead aircraft has been 
met (determined through either a generic rule -based 
spacing procedure, i.e., safe for all combinations of 
aircraft performance, or by using an on-board self- 
spacing tool). For SATS departures, pilots will file 
flight plans with a SATS departure procedure to a 
departure fix (DF, i.e.. Figure 1 Ellen or Ginny), 
obtain ATC clearance, and then use on-board 
information/tools to find a departure window, e.g., 
allowed to depart if there are no arriving aircraft 
within 5nm of the airport. The pilot would then 
depart and contact ATC according to the departure 
procedure. 
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Figure 2. HVO Validation 


HVO CONOPS Validation 

The SATS HVO CONOPS [4] was developed 
through a four phase, building block research 
process depicted in Figure 2. 

Phase one shows the CONOPS growing out of 
a need for developing the HVO “concept model” 
and documenting it. The key safety properties of a 
draft HVO CONOPS were also established by a 
mathematical verification method based on formal 
logic and theorem proving [5]. This study began 
formally verifying that self-separation is maintained 
when pilots adhere to the CONOPS procedures 
(including AMM logic). 

Phase two involved the development of a 
simulation environment (computer model) that 
included the AMM. The AMM was verified by 
testing its accurate function during a representative 
set of HVO scenarios. 

Phase three includes the bulk of the SATS 
HVO experimental work in validating the SATS 
HVO CONOPS (concept model) through human in 
the loop (HITL) studies. The HVO flight 
experiment was flown to validate pilot acceptability 
of a subset of the HVO simulation scenarios. Also, 
the ATC simulation study focused on determining 
controller acceptability of the concept model was 
completed in December 2004 [9]. 

Phase four was a proof-of-concept public 
demonstration of six SATSLab aircraft flying the 


SATS HVO CONOPS procedures in the 2005 
SATS Technology Demonstration held in Danville, 
Virginia. 

All four phases provided feedback to the 
improvement of the SATS HVO CONOPS and 
ultimately toward recommending a viable way to 
improve upon the one- in/one-out procedure in place 
in the NAS today. 

HVO Simulation and Flight 
Experiments 

Determining pilot acceptability of HVO meant 
investigating research objectives through a piloted 
simulation and a subsequent flight experiment : 

Comparing the SATS HVO CONOPS to the 
one-in-one-out procedural control environment 
available today (Baseline)... 

• Can pilots safely and proficiently fly the 
airplane while performing SATS HVO 
procedures? 

• Do pilots perceive that workload, while using 
HVO procedures and tools, is no greater than 
flying in today’s system? 

The analysis of flight technical error (FTE) 
data summarized in this paper compliments 
qualitative subject pilot assessments of their own 
workload, situation awareness, and HVO usability. 
Based on this FTE analysis, all evaluation subjects, 
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low-time instrument-rated pilots, flew the HVO 
procedures safely and proficiently in comparison to 
today’s system. FTE data were collected during the 
piloted HVO simulation and flight experiments 
completed in the summer of 2004. The HVO flight 
experiment was flown to validate pilot acceptability 
of a subset of the HVO simulation scenarios. A 
common pool of pilots was used. Fifteen pilots 
flew the HVO Simulation Experiment, and 12 of 
those pilots flew the HVO Flight Experiment. This 
reduced training requirements for the HVO Flight 
Experiment and allowed pilots to progress logically 
from hand-flying a medium fidelity general aviation 
(GA) simulator to the Cirrus SR22 aircraft. 

Hypotheses 

Flight Technical Error (Baseline & SATS 
scenarios) - the subject of this paper 

• Subject pilots will fly within the 
FAA’s Practical Test Standards (PTS) 
for the instrument rating 100% of the 
time during all scenarios 

• Deviations from assigned flight paths 
(i.e., RMSE values) will be equivalent 
across all scenarios 

Subjective Workload (Baseline & SATS 
scenarios) - reported in references [6-8] 

• Equivalent workload ratings will be 
associated with alj scenarios 

Situation Awareness (Baseline & SATS 
scenarios) reported in references [6-8] 

• Equivalent situation awareness ratings 
will be associated with all scenarios 

Procedure Conformance Monitoring (SATS 
scenarios only) reported in references [6-8] 

• Subject pilots will fly within 
conformance of the SATS HVO 
procedures 100% of the time 

Experiment Design and Procedure 

The experiment design matrix shown in Figure 
3 includes the five scenarios flown in the 
simulation. Three of these scenarios were repeated 
in the flight experiment. For the simulation 
experiment the experiment design used for data 


collection was a 2 (Procedure Type) x 5 (Scenario 
Type), within-subject design in which the same 15 
participants (i.e., low time instrument rated pilots) 
were assigned to each experimental cell (i.e., test 
condition). Simulation subject pilots 1 through 15 
(S i_i 5 ) were asked to perform all 10 test conditions 
in partially counterbalanced order under simulated 
IMC. For the flight experiment, a 2 (Procedure 
Type) x 3 (Scenario Type), within-subject design 
was used for data collection and 12 of the same 15 
subject pilots (F [_ 12 ) performed six test conditions 
twice in partially counterbalanced order under 
simulated IMC. Dependent measures included pilot 
FTE and subjective assessments of workload and 
situation awareness. 
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Figure 3. Experiment Design Matrix 
Scenarios 

The HVO experiments were designed to 
represent the HVO CONOPS through a sample set 
of scenarios : 

• Departure with approaching traffic - this 
scenario was flown in the simulation 
experiment only. The pilot task was to receive 
an ATC clearance, taxi onto the active runway, 
takeoff, and depart the airfield. In Baseline 
scenarios, ATC waited to give departure 
clearance to the subject pilot’s aircraft until an 
approaching aircraft had landed and canceled 
its clearance. In HVO, the pilot was to self- 
determine when to depart by finding an 
opening in the approach traffic flow. This was 
done by the pilot using a traffic display that 
showed aircraft on the approach and as long as 
the approaching aircraft was beyond the final 
approach fix (FAF), a departure was allowed. 
The durations of the SATS scenarios were 
about half as long as the durations of the 
Baseline scenarios. 
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• Approach without traffic (no holding 
required). The pilot task was to descend and 
fly the approach via ATC clearance during the 
Baseline scenarios. During the SATS 
scenarios, the pilot requested and received a 
“lateral entry, follow none” sequence from the 
AMM, was handed off into the SCA from 
ATC, and self- initiated the approach. 

Typically the durations of the SATS scenarios 
were equivalent to those of the Baseline 
scenarios. 

• Approach with virtual traffic (holding 
required). This scenario clearly differentiated 
Baseline from HVO. In Baseline, the pilot 
waited behind two other aircraft in holding 
until they had landed and ATC provided 
clearance to begin the approach (i.e., 30+ 
minutes in holding). In HVO, the pilot 
followed the AMM sequencing behind the two 
other aircraft, self-separated from the other 
aircraft in the SCA and self-initiated the 
approach by following advisories provided by 
a self-spacing software tool The SATS 
scenarios’ durations were about half that of the 
Baseline scenarios. 

• Approach to missed approach with virtual 
traffic (holding required). This scenario 
included having the pilot fly the missed 
approach. SATS scenarios required flying the 
missed approach while self-separating in the 
SCA. The SATS scenarios' durations were 


about two-thirds as long as the Baseline 
scenarios’ durations. 

• Multi-pilot linked simulation approaches 

(holding required). This scenario was flown 
with four linked simulators to verify the HVO 
concept by introducing the variability of 
individually flown aircraft (instead of using 
one pilot in a scenario with precisely flown 
virtual traffic). The goal was to determine if 
pilots could conduct HVO while serving as 
traffic for one another. Flown only in the 
medium fidelity simulation, pilots 
accomplished this task safely and proficiently. 
The time required to complete a multi-pilot 
HVO scenario was about one-quarter to one- 
third of the duration of a Baseline scenario. 

The experimental scenario flight profiles were 
based on GPS-T instrument approach procedures. 
Figure 4 depicts the progressive development of the 
experiment flight profile that began with the 
template SCA in the CONOPS and was 
implemented as shown in the simulation and flight 
approach charts. Subject pilots used these to fly 
experiment scenarios (Baseline charts not shown). 
The approach chart is a key element of the pilot 
procedures and coupled with task training, all 
evaluation pilots were able to fly the HVO 
procedures (described earlier with Figure 1) and the 
accompanying non-precision GPS-T instrument 
flight approach procedure. 
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Figure 4. HVO Procedure Development 

Subject Pilots and Training 

In choosing a subject pool to validate the HVO 
concept, researchers identified criteria for a pool of 
pilots who were capable, but not overly 
experienced. Although not tested, it is inferred that 
if the low-time instrument rated subject pool 
validated the concept positively (as was expected), 
then more experienced pilots would do the same. 
Subject pilot criteria were established to select low- 
time (less than 1000 flight hours) instrument rated 
pilots who were current to fly in IFR. None of the 
participants had previously flown a Cirrus SR22 
aircraft, worked as a flight crewmember for an air 
carrier within the last year, or flown for the 
military. 

A building block approach was employed to 
prepare subject pilots to accomplish the experiment 
tasks. This meant they manually flew the scenarios 
in simulated IMC using traditional round-dial 
instruments for primary flight guidance information 
(i.e., without autopilot). Advisory information was 


provided through a research multi- function display 
(MFD) that included a moving map with navigation 
information and traffic depiction. For the SATS 
scenarios, AMM sequence info, and procedure 
support information was also included on the MFD. 
Before the simulation experiment, subject pilots 
were first oriented to the HVO concept; and then 
they were given a session of hands-on simulator 
training to learn the simulator performance and 
controls. The research MFD interface required 
focused training to orient the subjects to its 
functionality and operation. Subjects were then 
given a more in-depth HVO research display 
training session that fully prepared them for the task 
of flying both Baseline and SATS scenarios. 
Through debriefing comments, all subject pilots 
agreed their training was adequate to perform 
simulation experiment tasks. 

To train pilots for the flight experiment, they 
were provided an electronic read-ahead training 
briefing before arrival and were given a 1 hour 
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training session that reviewed the briefing and 
provided them a hands-on simulator session to re- 
familiarize them with the experiment tasks. 
Orientation to the Cirrus SR22 was conducted by 
the safety pilot (a certified Cirrus instructor) on the 
ground and in flight. In flight training included 
aircraft basic instrument maneuver training and two 
practice GPS instrument approaches before data 
runs began. Through debriefing comments, all 
subject pilots agreed that their training was 
adequate to perform flight experiment tasks. 

Experiment Platform and Profiles 

Figure 5 shows the GA simulator and the 
Cirrus SR22’s instrument panel used for the 



Simulation Experiment Platform 
GA Simulator, similar to Cirrus SR22 


experiments. The simulator was a building block 
interface for the aircraft, and both used common 
research software to drive the MFD and the 
Horizontal Situation Indicator (HSI). Variation 
between the simulation and flight profiles was also 
deliberately minimized so as not to alter the 
experiment objectives or hypotheses. Pilot tasks 
(manual flight of arrival, holding, non-precision 
approach with respect to target airspeed, altitude, 
and lateral path) were to different target values for 
the simulation and flight, but in both cases pilots 
were tasked to meet instrument PTS criteria. 



Flight Experiment Platform 
Cirrus SR22 



Figure 5. HVO Experiment Platform Progression (note commonality of HSI and MFD) 


FTE Data Analysis 

Airspeed, altitude and lateral path deviation 
were measured during the experiments to assess the 
FTE of pilots flying Baseline and SATS approach 
scenarios (departure scenarios were not assessed by 
FTE measures). Their task objective was to fly the 
scenario while maintaining IFR PTS criteria [10]. 
FTE was measured as performance error from that 
task objective. Data were analyzed by way of 
repeated measures Analysis of Variance (ANOVA) 
tests, with the main effect of procedure type being 
of primary interest, and a 5 -percent significance 
level (i ,t.,p = 0.05) for the statistical analyses was 
set a priori [11]. 


While the hypothesis that subject pilots would 
“maintain instrument rating PTS 100% of the time 
during all scenarios” was not supported, the 
viability and acceptability of the SATS HVO 
concept is not diminished because in all cases the 
second hypothesis that “deviations from assigned 
flight paths (i.e., RMSE values) would be 
equivalent across all scenarios” was supported by 
the FTE results. In all cases, die pilots’ FTEs that 
occurred while flying SATS scenarios were not 
greater than the FTE’s that occurred while flying 
Baseline scenarios. In some cases, it was found 
through a statistical analysis of FTE data that their 
FTE during the SATS scenarios was significantly 
smaller (i.e., better) than during the Baseline 
scenarios. 
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Root Mean Squared Error (RMSE) with 
respect to Airspeed Target on the approach 

Pilots were instructed to fly the same target 
airspeeds for both the SATS and Baseline scenarios. 
Target airspeeds were chosen for the three segments 
of the GPS-T non-precision instrument approach 
procedure: initial (IAF to Intermediate Fix or IF); 
intermediate (IF to Final Approach Fix or FAF), 
and final (FAF to Missed Approach Point or MAP). 
The target airspeeds for the initial, intermediate and 
final segments were 110, 100 and 95 knots, 
respectively for the simulation experiment. The 
flight experiment used 120, 110, and 100 as target 
airspeeds. The different speeds were chosen for the 
flight experiment by the safety pilot to ensure better 
aircraft performance. Instrument rating PTS 
requires the pilot maintain +/- 10 knots of the target 
airspeed [10]. 

A significant difference was found to exist 
between the airspeed target RMSE values 
associated with each procedure type in the 
simulation experiment (p = 0.001). A RMSE 
airspeed deviation of 13.41 knots occurred during 
the Baseline scenarios [Mean (M) = 13.41, Standard 
Deviation (SD) = 7.65, Sample Size (N) = 60], and 
a RMSE airspeed deviation of 7.47 knots occurred 
during the SATS scenarios (M = 7.47, SD = 1.79, N 
= 60). 


A significant difference was also found to exist 
between the airspeed target RMSE values 
associated with each procedure type in the flight 
experiment (p = 0.003). A RMSE airspeed 
deviation of 5.82 knots occurred during the 
Baseline scenarios (M = 5.82, SD = 1.81, N = 72), 
and a RMSE airspeed deviation of 4.82 knots 
occurred during the SATS scenarios (M = 4.82, SD 
= 1.29, N = 72). 
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Figure 7. Flight Experiment Deviation from 
Airspeed Target 

This finding illustrated in figure 7 indicates 
that subject pilots maintained airspeed with respect 
to an assigned target value more accurately when 
they performed the SATS scenarios than when they 
performed the Baseline scenarios. 
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Figure 6. Simulation Experiment Deviation from 
Airspeed Target 

This finding shown in figure 6 indicates that in 
the simulation experiment, subject pilots maintained 
airspeed with respect to an assigned target value 
more accurately when they performed the SATS 
scenarios than when they performed the Baseline 
scenarios. 


Airspeed FTE Simula tion-to -flight 
conclusion: While the hypothesis that subject 
pilots would fly within the FAA’s PTS for the 
instrument rating 100% of the time during all 
scenarios was not supported by the airspeed target 
RMSE values collected during in the simulation 
experiment, the viability and acceptability of the 
SATS HVO concept is not diminished because the 
second hypothesis was supported in both of the 
experiments. Coupling the findings of the 
simulation and flight experiments indicates similar 
results for both experiments. Specifically, subject 
pilots maintained airspeed with respect to an 
assigned target value more accurately when they 
performed the SATS scenarios than when they 
performed the Baseline scenarios. Therefore, the 
results of the flight experiment validate the 
simulation experiment’ s results. 
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120 


Percent time within Altitude Envelope on 
the approach 

Pilots were instructed to fly within the same 
PTS altitude envelope for both the Baseline and 
SATS scenarios: -100 ft of “at or above” altitudes, 
and +100 and -Oft for the Minimum Descent 
Altitude (MDA) until MAP or visual transition to 
landing [10]. 

No significant difference was found to exist 
between the percentages of time within altitude 
envelope associated with each procedure type in the 
simulation experiment (p = 0.141). During the 
Baseline scenarios, subject pilots flew within the 
defined altitude envelope 84.41% of the time (M = 
84.41 SD = 17.61, N = 60). During the SATS 
scenarios, subjects flew within the defined altitude 
envelope 89.19% of the time (M = 89.19, SD = 
5.98, N = 60). 
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Figure 8. Simulation Experiment Percent Time 
in Altitude Envelope 

This finding shown in figure 8 indicates that in 
the simulation experiment subject pilots maintained 
altitude within an assigned envelope equally well 
when they performed the Baseline scenarios and 
when they performed the SATS scenarios. 

In the flight experiment, no significant 
difference was found to exist between the 
percentages of time within altitude envelope 
associated with each procedure type ( p = 0.300). 
During the baseline scenarios, subject pilots flew 
within the defined altitude envelope 97.42% of the 
time (M = 97.42, SD = 5.44, N = 72). During the 
SATS scenarios, subject pilots flew within the 
defined altitude envelope 98.41% of the time (M = 
98.41, SD = 3.58, N = 72). 



Procedure Type 


Figure 9. Flight Experiment Percent Time in 
Altitude Envelope 

This finding illustrated in figure 9 indicates 
that in the flight experiment subject pilots 
maintained altitude within an assigned envelope 
equally well when they performed the baseline 
scenarios and when they performed the SATS 
scenarios. 

Altitude FTE simulation-to -flight 
conclusion: Coupling the findings of the 
simulation and flight experiments indicates similar 
results for both experiments. The first FTE 
hypothesis that subject pilots would maintain 
instrument PTS 100% of the time was not supported 
in both experiments, but the second hypothesis was 
supported in both experiments. Specifically, 
subject pilots maintained altitude with respect to an 
assigned target value equally well when they 
performed the SATS scenarios and when they 
performed the Baseline scenarios. Therefore, the 
results of the flight experiment validate the results 
of the simulation experiment. 

RMSE with respect to Lateral Path Target 
on the approach 

Pilots were instructed to use target path as 
approach flight path during both the Baseline and 
SATS scenarios. Instrument PTS is to maintain 
within % scale deflection of the course deviation 
indicator (CDI) [10]. Full scale deflection on the 
research HSI represented .3 nautical miles through 
the approach procedure. 

A significant difference was found to exist 
between the lateral path deviation RMSE values 
associated with each procedure type in the 
simulation experiment {p = 0.037). A RMSE lateral 
path deviation of 0.12 nautical miles (nm) occurred 
during the Baseline scenarios (M = 0.12, SD = 0.19, 
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N = 60), and a RMSE lateral path deviation of 0.05 
nm occurred during the SATS scenarios (M = 0.05, 
SD = 0.03, N = 60). 


assigned target value more accurately when they 
performed the SATS scenarios than when they 
performed the baseline scenarios. 
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Figure 10. Simulation Experiment Deviation 
from Lateral Path Target 

This finding shown in figure 10 indicates that 
subject pilots maintained lateral path deviation with 
respect to an assigned target value more accurately 
when they performed SATS scenarios in the 
simulation experiment. 

In the flight experiment, a significant 
difference was found to exist between the lateral 
path deviation target RMSE values associated with 
each procedure type {p = 0.045). A RMSE lateral 
path deviation of 0.12 nm occurred during the 
baseline scenarios (M = 0.12, SD = 0.17, N = 72), 
and a RMSE lateral path deviation of 0.08 nm 
occurred during the SATS scenarios (M = 0.08, SD 
= 0.05, N = 72). 
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Figure 11. Flight Experiment Deviation from 
Lateral Path Target 

This finding as shown in figure 1 1 indicates 
that in the flight experiment, subject pilots 
maintained lateral path deviation with respect to an 


Lateral Path FTE simulation- to -flight 
conclusion: Coupling the findings of the 
simulation and flight experiments indicates similar 
results for both experiments supporting both 
hypotheses. The first hypothesis that pilots would 
maintain instrument PTS 100% of the time was 
supported as was the second hypothesis that 
deviations would be equivalent across all scenarios. 
Specifically, subject pilots maintained lateral path 
with respect to an assigned target path more 
accurately when they performed the SATS 
scenarios than when they performed the Baseline 
scenarios. Therefore, the results of the flight 
experiment validate the results of the simulation 
experiment. 


Conclusion: 

This paper provides an analysis of Flight 
Technical Error (FTE) from recent SATS 
experiments, called the Higher Volume Operations 
(HVO) Simulation and Flight experiments, which 
NASA conducted to determine pilot acceptability of 
the HVO concept for normal operating conditions. 
Reported are FTE results from simulation and flight 
experiment data indicating that the SATS HVO 
concept is viable and acceptable to low-time 
instrument rated pilots when compared with today’s 
system (Baseline). Described is the comparative 
FTE analysis of lateral, vertical, and airspeed 
deviations from the baseline and SATS HVO 
experimental flight procedures. 

Based on FTE analysis, all evaluation subjects, 
low-time instrument-rated pilots, flew the HVO 
procedures safely and proficiently in comparison to 
today’s system. Specifically, subject pilots 
maintained airspeed and lateral path more 
accurately when they performed the SATS 
scenarios than when they performed the Baseline 
scenarios. Subjects maintained altitude equally 
well in both SATS and Baseline scenarios. In all 
cases, the results of the flight experiment validated 
the results of the simulation experiment and 
confirm the utility of the simulation platform for 
comparative HITL studies of SATS HVO and 
Baseline operations. 
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The piloted experiments described in this 
paper were part of the building-block validation and 
verification process of the SATS HVO CONOPS 
that included multiple elements ranging from 
formal analysis of the procedures to flight test, to 
full-system architecture prototype that was 
successfully shown to the public at the June 2005 
SATS Technical Demonstration in Danville, VA. 
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